Skip to content

refactor(logstash): drop PKCS#8 filename shim, use native PKCS#8 generation#131

Merged
Oddly merged 5 commits intomainfrom
feat/logstash-drop-pkcs8-conversion
Apr 15, 2026
Merged

refactor(logstash): drop PKCS#8 filename shim, use native PKCS#8 generation#131
Oddly merged 5 commits intomainfrom
feat/logstash-drop-pkcs8-conversion

Conversation

@Oddly
Copy link
Copy Markdown
Owner

@Oddly Oddly commented Apr 13, 2026

The unencrypted PKCS#8 PEM key now lands directly at <hostname>.key across all three cert modes; the -pkcs8.key filename is gone. Standalone mode uses community.crypto.openssl_privatekey with format: pkcs8, which removes the openssl genrsa + openssl pkcs8 shell two-step. The ES_CA flow decrypts straight from the certutil unarchive into the target path instead of copy-then-convert. External mode just uses whatever filename you supplied.

Adds logstash_tls_copy_certs (default true, backward-compatible). Set to false in external mode to skip the copy and reference the operator-managed paths directly. That makes certmonger and cert-manager rotation work without an Ansible re-run because the renewal tool rewrites the files and restarts Logstash itself. A soft warning fires if the key is present but looks unreadable by the logstash group; no hard fail, since plenty of setups manage permissions differently.

Molecule verify now asserts the key header is -----BEGIN PRIVATE KEY----- so a regression to PKCS#1 or encrypted keys fails loudly. Docs updated: the TLS guide's trust-chain diagram, format table, and a new hands-off rotation section, plus the Logstash reference's key-format section rewritten with per-mode key provenance.

Closes #126.

Summary by CodeRabbit

  • New Features

    • Added logstash_tls_copy_certs to allow using externally provided certificates in-place for hands-off rotation.
  • Documentation

    • Added hands-off rotation guide, updated TLS trust-chain diagram, and clarified certificate/key naming and examples.
  • Improvements

    • Keys are now produced as unencrypted PKCS#8 natively and Logstash TLS configuration now references the configured cert/key paths with tightened verification and permissions.

…ration

The unencrypted PKCS#8 PEM key now lands directly at `<hostname>.key`
across all three cert modes; the `-pkcs8.key` filename is gone.
Standalone mode uses `community.crypto.openssl_privatekey` with
`format: pkcs8`, which removes the `openssl genrsa` + `openssl pkcs8`
shell two-step. The ES_CA flow decrypts straight from the certutil
unarchive into the target path instead of copy-then-convert. External
mode just uses whatever filename you supplied.

Adds `logstash_tls_copy_certs` (default `true`, backward-compatible).
Set to `false` in external mode to skip the copy and reference the
operator-managed paths directly. This makes certmonger and cert-manager
rotation work without an Ansible re-run because the renewal tool
rewrites the files and restarts Logstash itself. A soft warning fires
if the key is present but looks unreadable by the logstash group; no
hard fail, since plenty of setups manage permissions differently.

Molecule verify now asserts the key header is `-----BEGIN PRIVATE KEY-----`
so a regression to PKCS#1 or encrypted keys fails loudly. Docs updated:
the TLS guide's trust-chain diagram, format table, and a new hands-off
rotation section, plus the Logstash reference's key-format section
rewritten with per-mode key provenance.

Closes #126.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 13, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6d4e46e8-1dd4-43c7-a02e-e6c56e4eaa54

📥 Commits

Reviewing files that changed from the base of the PR and between de6865f and e184c37.

📒 Files selected for processing (1)
  • roles/logstash/tasks/logstash-security.yml

📝 Walkthrough

Walkthrough

This PR removes PKCS#8 conversion and standardizes Logstash private key names to <hostname>.key, adds logstash_tls_copy_certs to allow referencing external certs in place, updates task logic/templates to use computed input cert paths, updates test fixtures/molecule scenarios, and documents the changes in guides and changelog.

Changes

Cohort / File(s) Summary
Documentation & Changelog
CHANGELOG.md, docs/guide/tls.md, docs/reference/logstash.md
Document .key filename, new logstash_tls_copy_certs variable, hands-off rotation flow, updated key format/keystore behavior, and update TLS examples/naming.
Role Defaults
roles/logstash/defaults/main.yml
Add new default variable logstash_tls_copy_certs: true with explanatory comment.
Role Configuration Logic
roles/logstash/tasks/main.yml
Add set_fact task to compute _logstash_input_cert_path, _logstash_input_key_path, _logstash_ca_cert_path based on logstash_cert_source and logstash_tls_copy_certs; inserted before templates.
Role Security Tasks
roles/logstash/tasks/logstash-security.yml
Remove -pkcs8.key naming and intermediate conversion steps; generate/use unencrypted PKCS#8 PEM .key directly (standalone via community.crypto.openssl_privatekey); conditional copy vs stat/assert/debug when logstash_tls_copy_certs is false; update permissions handling.
Templates
roles/logstash/templates/10-input.conf.j2
Replace hardcoded per-host cert paths with _logstash_input_cert_path, _logstash_input_key_path, _logstash_ca_cert_path so inputs reference either copied files or original external paths.
Test Cert Generation
molecule/shared/generate_test_certs.yml
Generate CA and server private keys directly in PKCS#8 (format: pkcs8) and write server.key with correct permissions; remove conversion step and renamed outputs.
Molecule — External Scenario
molecule/logstash_ssl/converge.yml, molecule/logstash_ssl/verify.yml
Update expected key paths from -pkcs8.key.key; verification now asserts unencrypted PKCS#8 PEM header and correct permissions.
Molecule — Standalone Scenario
molecule/logstash_standalone_certs/converge.yml, molecule/logstash_standalone_certs/verify.yml
Update expected key paths from -pkcs8.key.key; add header and permission checks for generated keys.

Sequence Diagram(s)

mermaid
sequenceDiagram
rect rgba(200,200,255,0.5)
participant CertManager as Cert Manager (certmonger / cert-manager)
end
rect rgba(200,255,200,0.5)
participant FS as Filesystem (external cert path)
end
rect rgba(255,200,200,0.5)
participant Ansible as Ansible role (logstash)
end
participant Logstash as Logstash service

CertManager->>FS: write cert + key (e.g. /etc/pki/logstash/)
FS->>Ansible: Ansible sees `logstash_cert_source: external`
Ansible->>Ansible: evaluate `logstash_tls_copy_certs`
alt copy enabled
    Ansible->>FS: copy cert/key -> `{{ logstash_certs_dir }}/<host>-server.crt` and `<host>.key`
    Ansible->>Logstash: configure Logstash to use files in `logstash_certs_dir`
else copy disabled (hands-off)
    Ansible->>FS: stat provided paths; warn if unreadable/incorrect perms
    Ansible->>Logstash: configure Logstash to reference original external paths
end
CertManager->>Logstash: on renewal, restart post-save (no Ansible run needed when copy disabled)

mermaid
sequenceDiagram
rect rgba(255,230,200,0.5)
participant Ansible2 as Ansible (standalone flow)
end
participant Crypto as community.crypto.openssl_privatekey
participant FS2 as {{ logstash_certs_dir }}
participant Logstash2 as Logstash

Ansible2->>Crypto: request RSA private key (format: pkcs8)
Crypto->>FS2: write unencrypted PKCS#8 PEM -> `<hostname>.key` (mode/owner set)
Ansible2->>Logstash2: configure Logstash to use `<hostname>-server.crt` and `<hostname>.key`

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: dropping PKCS#8 filename shim and using native PKCS#8 generation across all cert modes.
Linked Issues check ✅ Passed All Phase 1 objectives from #126 are met: PKCS#8 conversion removed across external/elasticsearch_ca/standalone modes. Phase 2 objectives are fully addressed with logstash_tls_copy_certs variable enabling direct external cert usage.
Out of Scope Changes check ✅ Passed All changes align with #126 objectives: PKCS#8 elimination, native key generation, copy control variable, and related documentation/test updates. No unrelated modifications detected.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/logstash-drop-pkcs8-conversion

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
roles/logstash/tasks/logstash-security.yml (1)

9-14: ⚠️ Potential issue | 🟠 Major

Fail fast when hands-off TLS files are missing.

With logstash_tls_copy_certs: false, this branch only warns about key readability. Missing cert/key/CA paths still flow into the rendered input config, so the play succeeds and Logstash fails on restart. In particular, the computed CA path falls back to {{ logstash_certs_dir }}/ca.crt when logstash_tls_ca_file is unset, but this branch never creates or validates that file.

Possible guardrail
     - name: logstash-security | Check external key readability (hands-off mode)
       when: not (logstash_tls_copy_certs | bool)
       block:
+        - name: logstash-security | Stat external certificate in place
+          ansible.builtin.stat:
+            path: "{{ logstash_tls_certificate_file }}"
+          register: _logstash_external_cert_stat
+
         - name: logstash-security | Stat external key in place
           ansible.builtin.stat:
             path: "{{ logstash_tls_key_file }}"
           register: _logstash_external_key_stat
+
+        - name: logstash-security | Determine CA path used by the pipeline
+          ansible.builtin.set_fact:
+            _logstash_external_ca_path: "{{ logstash_tls_ca_file | default(logstash_certs_dir ~ '/ca.crt') }}"
+
+        - name: logstash-security | Stat external CA in place
+          ansible.builtin.stat:
+            path: "{{ _logstash_external_ca_path }}"
+          register: _logstash_external_ca_stat
+
+        - name: logstash-security | Validate external TLS files exist in place
+          ansible.builtin.assert:
+            that:
+              - _logstash_external_cert_stat.stat.exists
+              - _logstash_external_key_stat.stat.exists
+              - _logstash_external_ca_stat.stat.exists
+            fail_msg: >-
+              With logstash_tls_copy_certs: false, the referenced TLS files must
+              already exist on the managed node.

Also applies to: 61-83

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@roles/logstash/tasks/logstash-security.yml` around lines 9 - 14, The current
"logstash-security | Validate external certificate paths are provided" task must
fail fast when logstash_tls_copy_certs is false by asserting presence and
readability of all required TLS files; update that task (and the similar block
at the 61-83 region) to check logstash_tls_certificate_file,
logstash_tls_key_file and logstash_tls_ca_file (or the computed fallback {{
logstash_certs_dir }}/ca.crt) are defined and exist/readable (use
ansible.builtin.stat and assert on stat.exists/stat.readable) when
logstash_tls_copy_certs == false so the play fails early instead of rendering
invalid configs that break Logstash on restart.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@roles/logstash/tasks/logstash-security.yml`:
- Around line 323-331: The task "logstash-security | Generate server key" using
community.crypto.openssl_privatekey for path "{{ logstash_certs_dir }}/{{
inventory_hostname }}.key" will recreate keys if the existing file is a
different format; update that task to add format_mismatch: convert so the module
converts an existing private key to format: pkcs8 instead of generating a new
key (preserving the key fingerprint and certificate relationship) while keeping
the existing size, type, owner, group and mode settings unchanged.

---

Outside diff comments:
In `@roles/logstash/tasks/logstash-security.yml`:
- Around line 9-14: The current "logstash-security | Validate external
certificate paths are provided" task must fail fast when logstash_tls_copy_certs
is false by asserting presence and readability of all required TLS files; update
that task (and the similar block at the 61-83 region) to check
logstash_tls_certificate_file, logstash_tls_key_file and logstash_tls_ca_file
(or the computed fallback {{ logstash_certs_dir }}/ca.crt) are defined and
exist/readable (use ansible.builtin.stat and assert on
stat.exists/stat.readable) when logstash_tls_copy_certs == false so the play
fails early instead of rendering invalid configs that break Logstash on restart.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3fac2a2d-ebca-4eaa-80d6-01314f7a1606

📥 Commits

Reviewing files that changed from the base of the PR and between e5e17b5 and c8337c2.

📒 Files selected for processing (12)
  • CHANGELOG.md
  • docs/guide/tls.md
  • docs/reference/logstash.md
  • molecule/logstash_ssl/converge.yml
  • molecule/logstash_ssl/verify.yml
  • molecule/logstash_standalone_certs/converge.yml
  • molecule/logstash_standalone_certs/verify.yml
  • molecule/shared/generate_test_certs.yml
  • roles/logstash/defaults/main.yml
  • roles/logstash/tasks/logstash-security.yml
  • roles/logstash/tasks/main.yml
  • roles/logstash/templates/10-input.conf.j2

Comment thread roles/logstash/tasks/logstash-security.yml
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
roles/logstash/tasks/logstash-security.yml (1)

97-110: Broaden the hands-off warning predicate.

The check on Lines 108-110 will warn on valid layouts like root:logstash 0440 or a key owned by logstash with 0600. That makes the warning noisy even when Logstash can read the file. Consider keying this off owner/group and the relevant read bit instead of a small mode whitelist.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@roles/logstash/tasks/logstash-security.yml` around lines 97 - 110, The
warning is too strict because it whitelists modes rather than checking whether
the logstash user actually has read access; update the when predicate on the
"logstash-security | Warn if external key is not readable by logstash group"
task to: only run when the file exists and is not readable by the logstash user
or group by inspecting _logstash_external_key_stat.stat.pw_name and .gr_name and
testing the mode read bits (convert _logstash_external_key_stat.stat.mode to an
int with base=8 and check owner-read (0o400) if pw_name == 'logstash' or
group-read (0o040) if gr_name == 'logstash'); keep the logstash_tls_key_file and
logstash_tls_copy_certs variables unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@roles/logstash/tasks/logstash-security.yml`:
- Around line 273-286: The task "logstash-security | Decrypt key to unencrypted
PKCS#8 PEM" currently uses the ansible.builtin.command invoking "openssl pkcs8
... -in {{ logstash_certs_dir }}/{{ ansible_facts.hostname }}/{{
ansible_facts.hostname }}.key ... -out {{ logstash_certs_dir }}/{{
inventory_hostname }}.key" and is guarded by a creates: "{{ logstash_certs_dir
}}/{{ inventory_hostname }}.key" which prevents re-decryption on cert renewal;
remove the creates: line (the creates guard) from that task so the openssl pkcs8
command always runs and the private key is re-decrypted to match renewed
certificates.

---

Nitpick comments:
In `@roles/logstash/tasks/logstash-security.yml`:
- Around line 97-110: The warning is too strict because it whitelists modes
rather than checking whether the logstash user actually has read access; update
the when predicate on the "logstash-security | Warn if external key is not
readable by logstash group" task to: only run when the file exists and is not
readable by the logstash user or group by inspecting
_logstash_external_key_stat.stat.pw_name and .gr_name and testing the mode read
bits (convert _logstash_external_key_stat.stat.mode to an int with base=8 and
check owner-read (0o400) if pw_name == 'logstash' or group-read (0o040) if
gr_name == 'logstash'); keep the logstash_tls_key_file and
logstash_tls_copy_certs variables unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 5a6ba692-dd7f-4758-b286-2fd99723603c

📥 Commits

Reviewing files that changed from the base of the PR and between c8337c2 and d0328f7.

📒 Files selected for processing (1)
  • roles/logstash/tasks/logstash-security.yml

Comment thread roles/logstash/tasks/logstash-security.yml Outdated
Oddly added 2 commits April 14, 2026 13:06
…eadability check

Drop the `creates:` guard on the openssl pkcs8 decrypt task. The accompanying
server-cert copy is unconditional, so guarding the key decrypt left the renewed
cert paired with the previous key and Logstash refused to start until the next
manual cleanup.

Replace the mode whitelist on the hands-off external-key warning with a check
that mirrors what Logstash actually needs: owner/group ownership combined with
the corresponding read bit (or a world-read bit). The previous predicate fired
on perfectly valid layouts like `root:logstash 0440` and stayed silent on some
that wouldn't actually work.
ansible-lint flagged no-changed-when after the creates: guard was removed.
The openssl pkcs8 task only runs when the surrounding renewal block fires
(_logstash_needs_cert_renewal), and a fresh decrypt always replaces the
file, so changed_when: true is the honest signal.
@Oddly Oddly closed this Apr 14, 2026
@Oddly Oddly reopened this Apr 14, 2026
@Oddly Oddly added the ci:run Trigger gated pull request CI label Apr 14, 2026
@Oddly Oddly merged commit e236b4f into main Apr 15, 2026
128 of 129 checks passed
@Oddly Oddly deleted the feat/logstash-drop-pkcs8-conversion branch April 15, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:run Trigger gated pull request CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove unnecessary PKCS#8 key conversion for Logstash

1 participant